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ABSTRACT 



California's Employment Training Panel was created in 1982 
to work in partnership with the state's businesses, labor unions, and 
government to provide training funds to California businesses. In 1995, the 
Employment Training Panel's administrative practices, strategic planning 
efforts, and contracting policies and practices/procedures were reviewed 
through an audit process that included interviews with panel executives and 
other key staff and a review of a sample of 15 contracts awarded during 
fiscal years 1993-94 through 1995-96 and a sample of internal contracts. The 
panel was found to be handling many of its contracting and administrative 
responsibilities well; however, several problems/areas requiring improvement 
were identified. Among the audit's conclusions were the following: the panel 
inappropriately used a training contract to earmark $7 million for a specific 
group; the panel may be extending inappropriately the period for which funds 
are available to it and may be exceeding its administrative cap; the panel 
could improve its strategic planning efforts; and it should estimate training 
contract amounts more accurately. The panel generally concurred with the 
audit findings and agreed that improvements can be made in several areas as 
identified in the audit report. (MN) 
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The Governor of California 
President pro Tempore of the Senate 
Speaker of the Assembly 
State Capitol 

Sacramento, California 95814 
Dear Governor and Legislative Leaders: 

As required by Chapter 865, Statutes of 1995, the Bureau of State Audits presents its audit report 
concerning the Employment Training Panel’s (panel) contracting and administrative practices. 
This report concludes that the panel has achieved many of its training program responsibilities 
despite some administrative and planning problems. Specifically, we found that the panel 
accomplishes many of its program and administrative responsibilities. However, the panel could 
improve its strategic planning by stating its objectives in measurable terms and establishing 
performance benchmarks for its employees. Additionally, the panel needs to better estimate the 
amount of money its training contractors will earn each year, as the amounts that remain unspent 
are excessive. Finally, the panel has engaged in certain administrative practices that, although 
legal, appear to be improper, including extending the life of its appropriated funds, expending 
more than 15 percent of appropriated funds on administration, and encumbering funds to a 
contract from which the panel received no training services. 



Respectfully submitted, 




KURT R. SJOBERG 
State Auditor 
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Summary 
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Audit Highlights . . . 

The Employment 
Training Panel is 
handling many of its 
contracting and 
administrative 
responsibilities well. 

However, it: 

0 Could improve its 
strategic planning 

efforts. 

0 Needs to estimate 
training contract 
amounts more 

accurately. 

0 May be 

inappropria tely 
extending the amount 
of time funds are 
available to it, and 
exceeding its 
administrative cap. 

0 Inappropriately used 
a training contract to 
earmark $7 million 
for a specific group. 




Result is in Brief 

T he Employment Training Panel (panel) was originally 
created in 1982 to provide training funds to California 
businesses. The panel works in partnership with business, 
labor, and government to provide funds for training California's 
workforce in the skills necessary for businesses to remain viable 
and to compete in the global economy while providing workers 
with reasonable wages and secure employment. 

Our review focused on the panel's administrative practices, 
its strategic planning efforts, and its contracting policies, 
procedures, and practices. During our review, we found: 

• The panel is accomplishing many of its responsibilities. 
Specifically, it has issued contracts to only those companies 
eligible to receive training funds, developed a plan to 
adequately address a backlog of audits, and properly 
followed up on funds paid to, but not earned by, 
contractors. Additionally, the purposes of the panel's 
administrative contracts were reasonable; the processes the 
panel used to develop, implement, and communicate its 
directives, policies, and procedures were sensible; and 
its electronic data processing system is generally accurate. 

• The panel could improve its strategic planning efforts by 
stating its goals and objectives in measurable terms and 
identifying performance benchmarks for its employees. 
Although it identifies in its strategic plan several goals and 
objectives that are congruous with its mission, the panel has 
not stated them in measurable terms. Additionally, while 
most of its units produce reports that indicate their level of 
performance, they do not establish benchmarks against 
which they could evaluate employees' success in achieving 
goals and objectives. As a result, the panel is unable to 
impartially determine whether it and its employees 
successfully accomplish its goals and objectives and, 
therefore, fulfill its mission. 
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• The panel needs to better estimate the amount of money 
its training contractors will earn each year, as the amounts 
that remain unspent are excessive. During fiscal years 
1992-93 through 1994-95, contractors did not earn, and the 
panel was forced to disencumber or rescind, an average of 
42 percent of the funds it had awarded for training contracts. 
As a result, the panel unnecessarily tied up funds it could 
have provided to other contractors. 

• Some panel fiscal practices may not be appropriate. 
Specifically, by using a provision in the annual budget act 
that allows it to change prior-year appropriated funds into 
current-year funds, the panel may be inappropriately 
extending the amount of time funds are available to it. 
Additionally, because the Unemployment Insurance Code 
allows it to spend on its administration up to 15 percent of 
funds appropriated by the Legislature for a fiscal year plus 
up to 15 percent of those funds it disencumbers from prior 
fiscal years, the panel may be spending more than intended 
for administrative expenses. 

• The panel inappropriately encumbered $7 million for a 
contract from which it never intended to receive or pay for 
training services. During fiscal year 1993-94, the panel 
entered into a training contract with the California 
Manufacturers Association (CMA) for $7 million. However, 
it never intended the CMA to provide training services. 
Rather, the CMA was to develop, or market, potential 
projects from among CMA membership. Effectively, the 
panel earmarked $7 million in training funds for CMA 
members by encumbering these funds in the contract. 
As a result, it unnecessarily made some contractors wait 
for training funds, avoided reverting $2.9 million to 
the Unemployment Insurance Fund as required by the 
Unemployment Insurance Code, and wasted the time of its 
staff members, and the staff at the Employment Development 
Department who handle the panel's accounting function. 



Recommendations 



To ensure that it is successful in achieving its stated goals and 
objectives and fulfilling its mission, the panel should modify its 
strategic plan, stating its organizational goals and objectives in 
measurable terms whenever possible. Additionally, the panel 
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should establish benchmarks for key performance indicators for 
its employees to measure their success and challenge their 
performance. 

To more effectively manage the distribution of training funds, 
the panel needs to more accurately calculate training contract 
awards. In addition, the panel should include in its annual 
report the effect of implementing recommendations from the 
disencumbrance report. If the changes made are not effective 
in reducing disencumbrances, the panel should consider 
alternative means to better manage training funds, such as 
using phased contracts, imposing a penalty on unsuccessful 
companies, or other suggestions from the Advisory Research 
Council or the California State University, Northridge. 

To ensure the panel is spending its funds appropriately, the 
Legislature should consider the methods by which the panel is 
able to extend the life of its appropriated funds, and, if contrary 
to legislative intent, eliminate the budget act language allowing 
it to re-encumber funds. Further, the Legislature should 
consider the methods employed by the panel to use more than 
15 percent of appropriated funds for administration, and if 
contrary to the intent of the Legislature, limit administrative 
costs to a certain percentage of funds appropriated each year 
excluding amounts disencumbered from prior fiscal 
year appropriations. 

To improve the effectiveness and propriety of its operations, 
the panel should stop entering into contracts and encumbering 
training funds for which contractors do not intend to 
provide training services. 



Agency Comments 

The panel generally concurs with the findings and agrees that 
improvements can be made in several areas as identified in the 
audit report. In addition, the panel noted that it has already 
begun to address some of our recommendations. With regard to 
the issue of some fiscal practices being inappropriate, the panel 
believes that the Legislature did intend for the panel to have the 
ability to prolong the availability of funds beyond three years. 
Finally, the panel believes that the Unemployment Insurance 
Code allows it to include funds disencumbered from prior years 
when determining the amount it can spend on administrative 
costs. 
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Introduction 



Background 

T he Employment Training Panel (panel) was originally 
created in 1982 to assist employers in jeopardy of laying 
off workers or in need of skilled employees to work with 
new technologies. The panel provided funding to train 
employees in the use of new technologies, which, in turn, 
provided the employers with operating efficiencies. Effective 
January 1, 1994, legislation required the panel to focus on 
projects that benefit the California economy as a whole, rather 
than individual entities, by funding contracts that retrain 
workers in companies challenged by out-of-state competition. 

The panel consists of eight members who have backgrounds in 
business, management, labor, or employee relations. The 
governor appoints three members, including the chairman, 
while the Senate and Assembly each appoint two members. 
The eighth member, either the Secretary of the Trade and 
Commerce Agency or his or her designee, serves as an 
ex officio voting member. The panel's headquarters is located 
in Sacramento, and it has field offices in Sacramento, 
San Mateo, Los Angeles, and San Diego. The administrative 
head is the executive director, who oversees approximately 
128 employees. The panel is funded through a one-tenth of 
one percent tax imposed on employers subject to the 
unemployment insurance tax of California. 

The California Unemployment Insurance Code directs the panel 
to fund only those training projects that do the following: 

• Foster creation of high-wage, high-skilled jobs or retention 
of high-wage, high-skilled jobs that are threatened by 
out-of-state competition. 

• Encourage industry-based investment in human resource 
development that promotes the competitiveness of California 
industry through productivity and product quality 
enhancements. 

• Result in secure jobs for those who successfully complete 
training. 
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• Supplement rather than displace funds available from other 
programs. 

During fiscal year 1995-96, the panel entered into 226 training 
contracts totaling $48.8 million. Most of the projects it funds 
are to train new, recently employed, or incumbent workers of 
companies locating to or expanding operations in California, 
and to maintain the viability of current businesses. Contracts 
may extend for up to two years, and reimbursement is 
contingent upon successful contractor performance. 

These performance-based contracts require contractors to 
provide the agreed training, and require the employers to pay 
the minimum wage established in the contract and retain the 
trainee as an employee for a period of at least 90 days after 
the training is complete, to be reimbursed. When it enters into 
a contract with a company to provide training, the panel 
allocates, or encumbers, the total amount from the funds 
available through its annual appropriation from the State. This 
encumbrance reserves the funds so that the panel can pay the 
contractor if the contracting organization provides training that 
complies with applicable contract terms. If the contractor is not 
able to provide the training or does not provide a training 
program that meets the contract terms, the panel does not pay, 
but rather rescinds, or disencumbers, these funds. It may then 
commit the disencumbered funds to other contracts. 



The PaneVs Contracting Process 

To disburse training funds, the panel follows a contracting 
process that involves multiple units within the panel. Figure 1 
on page 3 depicts this process. It begins with the marketing unit 
generating interest in the panel's program and purpose. To 
inform eligible companies about the program and the funds 
available to them, the marketing unit works with the California 
Trade and Commerce Agency, which has an interagency 
agreement to perform marketing services for the panel. The 
California Trade and Commerce Agency has four regional 
marketing representatives throughout the State who work with 
the panel to apprise California businesses of the funding 
program and assist those businesses in submitting applications. 
A company wishing to receive panel funds must submit an 
application to the headquarters in Sacramento. Headquarters' 
staff perform an initial review to ensure that the application is 
complete and then assign it to one of the four regional offices. 




CALIFORNIA 





At the regional office, staff in one of the panel's development 
units work with the company to prepare a proposed agreement. 
Additionally, companies sometimes use the services of an 
outside consultant to help them complete their application, 
develop a proposed agreement, and administer their training 
program. When a proposed agreement is complete, the 
eight-member panel or the executive director evaluates it for 
possible approval. Currently, the executive director has 
discretion to approve most proposed agreements valued at 
$100,000 or less. The eight-member panel approves contracts 
over $100,000. When a proposed agreement receives 
approval, it becomes a contract. Once signed, one of the two 
monitoring units oversees the activities of the contractor to 
ensure compliance with the terms of the contract. In addition, 
the panel's audit unit performs reviews of selected contracts 
after they are completed to ensure that contractors provided 
training in accordance with the terms of the contract. 



Figure^ 1 

Employment Training Panel 
Contracting Process 
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Other Related Reviews 

the Panel Current^ Under Way 

In addition to our review, two other entities are currently 
reviewing the panel's operations. The panel has assembled the 
Advisory Research Council, consisting of panel members and 
individuals from labor, management, and industry, who will 
issue a report to the Legislature by December 31, 1996, on 
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ways the panel can improve its process of preparing, funding, 
and implementing contracts. Additionally, the panel has 
entered into an interagency agreement with California State 
University, Northridge, to study the effects the contractors' use 
of consultants has on the success of contracts. As described in 
the following section, we limited the scope of our audit in these 
areas so that we did not duplicate the efforts of these other 
reviews. 



Scope and Methodology 

Chapter 865, Statutes of 1995, directed the Bureau of State 
Audits to conduct a comprehensive management review and 
evaluation of the panel. This legislation directed that we 
include the following issues in our review: 

• The panel's contracting policies, procedures, and 
regulations; 

• The panel's decision making processes for contracts; 

• The cost and efficiency of the administration of internal 
panel contracts; 

• The methods and procedures involved in setting the panel's 
internal administrative and procedural regulations; and 

• The internal administrative policies and procedures of the 
panel. 

In addition to evaluating these issues, we determined whether 
the panel had established mission statements, identified goals 
and objectives, and developed workload standards for its 
various units. To accomplish this, we interviewed panel 
executives and other key staff. We also reviewed its most 
recent strategic plan to determine whether the plan identified 
goals and objectives, action plans, and time lines management 
could use as guides in accomplishing the panel's overall 
mission. 

To assess contracting policies, procedures, and regulations, we 
interviewed panel staff and identified the activities of the 
application review, contract development, monitoring, and 
audit units. Further, we determined whether any other entities 
had performed or were currently performing related reviews. 
We determined that two reviews currently underway address 
the panel's contract processes. As discussed in the previous 
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section, the Advisory Research Council and California State 
University, Northridge, are evaluating the panel's contracting 
practices. The Advisory Research Council is reviewing the 
panel's processes, policies, and regulations related to preparing, 
funding, and implementing training contracts. California State 
University, Northridge, is studying the effect the contractors' use 
of consultants has on the success of training contracts. 
Therefore, our audit did not include a review of these areas. 
However, we did evaluate the activities of the panel's audit 
unit. Specifically, to determine if they were reasonable and 
appropriate, we assessed the unit's audit sampling techniques 
and its plan to retire a backlog of audits accumulated at 
June 30, 1996. 

To assess its process for approving and awarding contracts, we 
interviewed key panel staff. We selected a sample of 15 
contracts awarded during fiscal years 1993-94 through 1995-96 
to determine if each contractor was eligible to receive panel 
funds. 

To evaluate the efficiency of administration of internal, or 
non-training, panel contracts, we reviewed a sample of internal 
contracts and determined whether the nature of the services or 
goods received was reasonable. 

To assess the methods and procedures involved in setting 
internal administrative and procedural directives, we 
interviewed key panel staff and learned how the panel develops 
its administrative and procedural directives. 
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Chapter 

The Panei Achieves Many of Its Responsibilities; 

However, It Could Improve Its 
Administration and Planning 



Chapter^ Summary 

T he mission of the Employment Training Panel (panel) is to 
work in partnership with business, labor, and government 
to provide funds for training California's workforce in the 
skills necessary for businesses to remain viable and to compete 
in the global economy while providing workers with reasonable 
wages and secure employment. Our review of several of the 
panel's operations revealed no significant problems in its efforts 
to fulfill this mission. For example, we found that the panel 
issued contracts only to those companies eligible to receive 
training funds, developed a plan to adequately address a 
backlog of training contractor audits, and adequately followed 
up on funds paid to, but not earned by, contractors. 
Additionally, the purposes of the panel's administrative 
contracts were reasonable. Moreover, the processes it used to 
develop, implement, and communicate its directives, policies, 
and procedures were sensible. Finally, the panel's electronic 
data processing system is generally accurate. 

Although we found no significant problems to report in the areas 
described above, the panel could improve its strategic planning 
efforts and its process for estimating amounts contractors will 
earn. First, the panel needs to complete its strategic planning 
efforts. Although it has defined its vision and mission statements 
and established goals and objectives to fulfill this mission, the 
panel has not stated them in measurable terms. Furthermore, 
some units have not set benchmarks against which they can 
evaluate their employees' success. Because the panel did not 
state its goals and objectives in measurable terms, and because 
several of its units have not identified benchmarks for their 
employees, the panel cannot determine impartially whether it is 
successfully accomplishing its goals and objectives and thus 
fulfilling its mission. 

Second, the panel could improve the way it determines how 
much money its training contractors will be able to earn each 
year. Specifically, the panel needs to better estimate and 
monitor the amount of funds it dedicates to individual 
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contractors, as the amounts that remain unspent are excessive. 
During fiscal years 1992-93 through 1994-95, contractors did 
not earn, and the panel was forced to disencumber or rescind, 
an average of 42 percent of the funds it had awarded for 
training contracts. For example, the panel disencumbered, or 
rescinded, $38 million (41 percent) of the $93 million it 
awarded in fiscal year 1993-94 because contractors did not 
provide the necessary training to fully earn the amount of funds 
allocated to them. As a result, the panel unnecessarily tied up 
funds it could have provided to other contractors. 

Finally, we identified some fiscal practices that may not be 
appropriate. First, by using a provision in the annual budget 
act, the panel is able to extend the amount of time funds are 
available to it by disencumbering and then immediately 
re-encumbering them. Second, because the Unemployment 
Insurance Code allows it to spend on administration up to 
15 percent of funds appropriated to it by the Legislature for a 
fiscal year plus up to 15 percent of those funds it disencumbers 
from prior fiscal years, the panel may be spending more than 
intended for administrative expenses. Finally, the panel 
inappropriately encumbered $7 million to a training contract 
from which it never intended to receive or pay for training 
services. As a result, some contractors had to unnecessarily 
wait for funding, the panel kept $2.9 million that should have 
reverted to the Unemployment Insurance Fund, and panel staff, 
as well as staff at the Employment Development Department, 
wasted contract and fiscal management efforts. 



The Panel Is Accomplishing Many of 
Its Responsibilities but Can 

Improve Its Strategic^ Planning Process 

The panel is accomplishing its responsibilities in many of the 
areas we reviewed. Specifically, we analyzed efforts in several 
program and administrative areas for providing training funds to 
eligible companies and found the panel was fulfilling many of 
its responsibilities. However, it could better state the goals and 
objectives listed in its strategic plan and identify benchmarks for 
its employees. In doing so, the panel could determine 
objectively whether it is successful in fulfilling its mission. 
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The Panel Is Fulfilling Its 
Responsibilities in Program Areas 

The panel is meeting many of its responsibilities related to 
funding training programs. For example, the panel adequately 
carried out its duties for ensuring that it issued contracts to only 
eligible companies. We reviewed a sample of 5 contracts the 
panel entered into for each of the last three fiscal years to 
determine if the companies receiving funds met the eligibility 
requirements established in the Unemployment Insurance Code. 
For all 15 contracts, the contractors met minimum eligibility 
standards. 




The panel has issued 
contracts only to eligible 
companies, has 
developed a plan to 
eliminate its audit 
backlog, and has 
followed established 
procedures to collect 
amounts due. 
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Additionally, the panel developed a plan to adequately address 
a backlog of training contract audits over the next three years. 
Before fiscal year 1995-96, it entered into an agreement with 
the Employment Development Department to audit completed 
contracts. In fiscal year 1995-96, using its own audit unit, the 
panel began performing these audits; however, due to staff 
shortages and the initial training of new auditors, it was not able 
to complete by June 30, 1996, 35 of the 47 audits it planned for 
fiscal year 1995-96. However, for 14 of these 35 audits, the 
panel had completed its audit fieldwork and only needed to 
issue a report. We reviewed the approved plan and determined 
that it appears to reasonably and adequately eliminate the 
backlog. 

Finally, the panel properly followed up on funds paid to, but not 
earned by, contractors. Most of these receivables resulted from 
the panel making advance payments to contractors and later 
determining, through its monitoring or audit functions, that the 
contractor did not earn the amount paid. We reviewed nine 
contracts for which the contractor owed the panel and found 
that it followed established collection procedures or was 
actively pursuing the amounts due. Despite its best efforts, 
however, the panel has been unsuccessful in collecting the 
amounts due for many of the contracts it overpaid. Specifically, 
during fiscal year 1995-96, contractors owed an average of 
$7.8 million. During this period, the panel only collected 
$370,000. For $5.1 million (65 percent) of the $7.8 million 
owed at June 30, 1996, the panel had exhausted its avenues for 
recovering the funds. These accounts are currently being 
processed through the Board of Control as uncollectable, are in 
bankruptcy, or have been referred to the Attorney General's 
Office for legal action. 
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The Panel Is Fulfilling Its Duties 
In Administrative Areas 

In many administrative areas, the panel is also accomplishing its 
responsibilities. For example, the methods it uses to identify 
issues and obtain information before establishing policies 
are reasonable and appropriate. In addition, the panel's 
non-training contracts involved services and goods that 
appeared appropriate and necessary for the panel to accomplish 
its mission. 

Furthermore, the panel's electronic data-processing system is 
generally accurate. Specifically, we reviewed a total of 
60 projects approved during fiscal years 1994-95 and 1995-96 
by comparing key information contained in the system to 
supporting documents in the panel's contract files. Although we 
found minor inaccuracies in the data and other minor 
weaknesses in the system, these errors did not materially affect 
the integrity of the information maintained by the 
data-processing system. 



The Panel Could Improve Its 
Strategic Planning Efforts 

As part of our audit of operations, we reviewed the panel's effort 
to adopt and implement a strategic plan. Although it has 
established both a vision and a mission statement for its 
organization, the objectives identified to fulfill its mission are 
not stated in measurable terms. Furthermore, the panel has not 
established performance benchmarks for some of its employees. 

Strategic planning is a long-term, future-oriented process of 
assessment, goal setting, and decision making that maps an 
explicit path between the present and a vision for the future. 
Essential elements of sound strategic planning include the 
following: 

• Analyzing the work environment to identify strengths, 
weaknesses, problems, and opportunities; 

• Identifying key issues relating to the organization's mission 
and planned activities; 

• Defining the mission and formulating consistent goals; 

• Establishing priorities among the goals and allocating 
resources accordingly; 
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• Defining the objectives necessary to achieve each stated 
goal; 

• Establishing time lines and action plans to complete each 
objective; 

• Defining benchmarks or targets for each significant activity; 
and 

• Measuring the results of planned operations against the 
benchmarks to evaluate performance and reset targets as 
necessary. 

Successful planning efforts provide many benefits both to an 
agency and to the clients it serves. For example, strategic 
planning will improve an agency's ability to anticipate and 
accommodate the future by identifying its issues, opportunities, 
and problems. Because it focuses on results, good strategic 
planning will enhance decision making at both the operational 
and executive levels. Planning efforts geared towards outcomes 
or benefits also explicitly emphasize client satisfaction. 
Successful strategic planning provides needed information to 
guide managers in making resource allocation decisions and 
establishes a basis for measuring the success of the agency's 
activities. 







Although the objectives 
identified in the panel's 
strategic plan are 
consistent with its 
mission, they are not 
stated in terms that can 
be measured or 
benchmarked. 




The fundamental concept underlying strategic planning is its 
dynamic nature. The planning process is not a one-time project 
that the organization completes. Strategic planning should be 
an iterative process that an agency refines and refocuses as the 
agency measures performance, resets targets, and gathers new 
information. 

In its strategic plan, the panel has established both a vision and 
a mission statement for its organization. The plan identifies the 
strategies and objectives the panel will employ to fulfill its stated 
mission. Additionally, each unit has a stated mission that is 
congruous with the panel's overall mission. The methods and 
objectives the panel has identified in its current strategic plan 
are also consistent with its mission statement. 

However, the objectives are not stated in measurable terms so 
that the panel can determine its success after the planning 
period is over. For example, one objective the panel includes 
in its current strategic plan is "ongoing introduction of the 
(panel) program to the public and provision of technical 
assistance prior to contract development." The plan goes on to 
explain that the panel will increase efforts to ensure that 
contractors have better up-front knowledge of all training 
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options and requirements to improve their training projects' 
chances for success. However, the panel has not identified 
how it will measure whether it is achieving this objective and, 
therefore, is unable to determine its success. As a result, the 
panel is unable to ascertain whether it needs to modify its 
strategies for fulfilling its mission. 

Additionally, we reviewed 5 of the panel's 1 1 units to determine 
if each is identifying performance indicators, such as 
benchmarks, time lines, or workload standards, and whether it 
is using the indicators to evaluate unit and employee 
effectiveness in achieving the unit's stated mission. While most 
of them had reports that showed the activities and production 
output, the units often had not identified, or did not use, 
benchmarks or standards against which their employees could 
be evaluated. Of the five managers we interviewed, three 
stated that, although they reviewed activity reports at least 
monthly, they had not established, or did not use, performance 
benchmarks to determine whether employees were performing 
adequately. 



The Panel Needs To Better Manage 

Its Distribution o /* Funds 

During our review, we noted areas in which the panel could 
improve its operating effectiveness and efficiency. Specifically, 
it could better estimate the funds each contractor will earn, 
thereby making funds available to more contractors than it does 
currently. Further, the panel needs to improve some of its fund 
management practices. 



The Panel Needs To Better Estimate the 
Amount of Funds Contractors Will Earn 

The panel needs to develop better estimates of the amounts 
its contractors will earn so that it can increase the number of 
contractors it serves in a particular fiscal year. By increasing 
the number of contractors, the positive impact training funds 
will have on the State's economy may occur earlier than if some 
contractors wait to receive funds. In addition, more accurate 
estimates of contractors' funding needs would lower the rate of 
disencumbrances the panel incurs each year. 
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The panel rescinded 
42 percent of the 
$90.6 million it awarded. 




When it does not accurately estimate the amount of funds a 
contractor will earn and awards too much money, the panel 
may force other contractors to wait for funds to train their 
employees. This delay then postpones the training funds' effect 
on the State's economy. For example, if we assume that 
demand for panel funds exceeds funds available for fiscal year 
1996-97, some contractors must wait until the panel receives 
additional funds in fiscal year 1997-98 and delay employee 
training. If the panel better estimated amounts required for 
contracts, it would be able to fund these delayed contracts 
sooner, thus minimizing the time required for panel funds to 
impact the State's economy. 

We reviewed disencumbrances for funds originally contracted 
during fiscal years 1992-93, 1993-94, and 1994-95, and found 
that the rate at which the panel disencumbered or rescinded 
money from individual contracts was exceedingly high. As 
Figure 2 below indicates, the panel disencumbered an average 
of $38.2 million, or 42 percent, of the $90.6 million it awarded 
for training contracts during these years. Even though the 
panel's disencumbrance rate has been decreasing, we estimate 
that it will still disencumber $28.3 million (34 percent) of the 
$83.6 million it awarded in fiscal year 1994-95. 



Figure. 2 

Panel Disencumbrances 




Although it subsequently commits disencumbered funds to other 
training contracts, this high rate of disencumbrances adversely 
affects the panel's program. When it disencumbers training 
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When training contracts 
are overfunded, the 
benefits to the State's 
economy from the 
unspent funds are 
delayed. 




funds and then awards them to a different contractor, the impact 
of those funds on the State's economy is delayed. The panel 
disencumbers most of the funds in question during the second 
year after contract approval. Therefore, if it entered into a 
contract in fiscal year 1994-95, most of the unearned funds from 
that contract will be disencumbered during fiscal year 1996-97. 
It will encumber the unearned funds to another contract, 
whose term will likely extend from fiscal year 1996-97 
through 1998-99. Even though the funds are re-encumbered in 
1996-97, they may not actually be spent until the end of the 
new contract period. As a result, funds the panel originally 
encumbered in fiscal year 1994-95 would not be spent until 
four years later, delaying the positive impact the training funds 
may have on the State's economy. While the above example 
assumes that all of the money re-encumbered in fiscal year 
1996-97 will be earned by and paid to the contractor, past 
history indicates that, on average, 42 percent of the panel's 
contract amounts are later disencumbered; therefore, a 
portion of these funds would again be disencumbered and 
re-encumbered to another new contract. This process could go 
on for several years, further postponing the impact on the State's 
economy of the appropriation the panel received in fiscal year 
1994-95. 



By Revising Its Contract Management 
Practices, the Panel Could Serve 
Employers Sooner 





The panel could reduce 
the amount of unspent 
training funds by 
addressing eight 
influential factors. 




In an effort to improve its program operations, the panel's 
planning and research unit conducted a study to determine the 
major causes of disencumbrances in training contracts and 
make recommendations for reducing them. In May 1995, the 
planning and research unit issued a report based 
on the study identifying eight factors that influence 
disencumbrances: 

• Larger contracts tend to have larger disencumbrance rates. 

• Contracts without amendments have higher disencumbrance 
rates than those with amendments. 

• Service industries have higher disencumbrance rates than 
manufacturing. 



• Newer businesses (less than ten years old) have 
higher disencumbrance rates than established businesses. 
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• Contractors that use subcontractors have higher 
disencumbrance rates than those that do not. 

• Contracts that include structured on-site training have 
higher disencumbrance rates than those that do not. 

• Contracts with more than 200 hours of training per trainee 
have higher disencumbrance rates than those with fewer 
hours. 

• Lack of panel administrative oversight in the early phases of 
a contract results in higher disencumbrances. 

The report also included six specific recommendations for 
reducing disencumbrances and acknowledged that the panel is 
not able to control some of the factors affecting the 
disencumbrance rate. Therefore, the panel may never be able 
to reduce its disencumbrances to zero. However, the report did 
state, and we agree, that the panel could lower the rate of 
disencumbrances by addressing those causes it can control. 

The panel has implemented four of these six recommendations. 
Specifically, it streamlined the original amendment process so 
the contractor can more easily amend its contract after 
execution if the need arises. In addition, the panel's 
development analysts now review proposed agreements, and 
identify and include in the package for approval the following 
risk indicators for all retraining contracts: 

• Contractors with previous contracts that resulted in large 
disencumbrances; 

• Proposals that include subcontractors that will perform 
administrative duties; 

• Contractors with more than 250 employees; 

• Training projects in the services industry; 

• Contractors in business ten years or less; 

• Proposals that include subcontractors for training, or for 
training and administration; 

• Proposals for more than 200 training hours per trainee; and 

• Proposals that include structured on-site training. 
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To simplify structured 
on-site training contracts, 
the panel revised its 
documentation 
requirements. 




In response to another recommendation, during fiscal year 
1996-97, the panel revised documentation requirements with 
which contractors must comply to be reimbursed for structured 
on-site training. Under the new requirements, rather than 
documenting the amount of structured on-site training each 
trainee receives, each contractor must report the number of 
hours its trainer spends with trainees and certify that each 
employee the contractor claims as trained has attained 
competency as specified by the curriculum. 



The final recommendation the panel implemented was 
establishing a control to determine whether a contractor is 
making progress. Approximately 30 days after the start of 
training, analysts conduct the first monitoring site visit. A 
typical site visit includes a review of program status, observation 
of training, trainer and trainee interviews, and a review of 
trainee records. The monitors normally conduct subsequent 
visits on a quarterly basis. If they identify projects for which no 
activity has occurred within six months after start-up, the 
monitors will offer immediate technical assistance to help 
initiate project activity. If no subsequent activity occurs within 
an additional three months, the monitor will terminate the 
project. 



Because disencumbrances often do not occur until two years 
after the panel awards a contract, and because the panel 
implemented these recommendations during fiscal years 
1995-96 and 1996-97, we were unable to determine the effect 
of the changes on the rate of disencumbrances. However, the 
panel should begin to see their effects by the end of fiscal 
year 1 996-97, and to a greater extent by the end of fiscal year 
1997-98. Finally, although it takes up to two years for most 
unearned funds to be disencumbered, the changes should lower 
the panel's disencumbrance rate to a limited degree. 



The panel has not implemented two of the six recommendations 
made in the disencumbrance report: implementing projects in 

phases and involving contractors in the administration of their 
own contracts. According to the executive director, the panel 
has not implemented contract phasing because of concerns 
raised by the Employment Development Department and 
the Department of Finance about making commitments for 
future years without placing funds aside. The executive director 
also stated that the panel's concerns regarding increased 
administrative workloads inhibited further consideration of 
contract phasing. He further stated that streamlining the 
amendment process by which a contractor increases or 
decreases the funding in a contract should reduce the tendency 
of contractors to ask initially for more money than they can 
realistically use. 
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The panel has 
implemented several 
measures to better 
manage its training 
contracts. 




The panel has already taken some measures to get the 
contractors' management involved in the administration of 
training programs. For example, it requires contractors to have 
one of their senior management representatives present at initial 
planning meetings. However, according to its executive 
director, the panel is not legally able to mandate that 
contractors manage their own training program administration 
or curricula and cannot prevent them from choosing to use 
subcontractors for these purposes. 

The Unemployment Insurance Code, Section 10205(g), requires 
the panel to evaluate the effectiveness of its training programs. 
In fulfilling this requirement, the panel entered into an 
interagency agreement with California State University, 
Northridge, to study the role of consultants in contracts. 
California State University, Northridge, will present its findings 
in December 1996. The panel should then submit these 
findings to the Legislature along with its plans to further involve 
contractor management in contract administration and 
performance. 



The Panel Could Adopt Policies To 
Deter Contractors From Overstating 
Their Funding Needs 

In addition to more closely estimating contract amounts, the 
panel could adopt measures that would discourage a contractor 
from requesting more training funds than it needs. The panel 
later disencumbers these excess funds when it determines the 
contractor will not be able to earn them. If it were to adopt 
policies that would deter contractors from overstating their 
needs, the panel could reduce disencumbrances. 

One way the panel could encourage contractors to make 
realistic funding requests would be to adopt a policy of 
penalizing those that do not successfully complete their 
contracts. For example, if a contractor does not successfully 
complete all the training for which it contracted, the 
panel would fund the completed training at a reduced rate. 
This would provide a fiscal incentive to the contractor to 
accomplish all the training planned and a disincentive to 
request more funds than necessary. In those instances in which 
a contractor earns no panel funds, and therefore the panel 
cannot reduce the amount otherwise funded, this option may 
not be possible. However, as indicated in Table 1 , according to 
the panel's disencumbrance report based on fiscal year 1993-94 
data, only 9.8 percent of the contracts resulted in 100 percent 
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disencumbrances. Therefore, this proposed policy would affect 
most contractors that do not complete the training outlined in 
their contracts. 



Table 1 

Success Rates of Fiscal Year 1993-94 
Panel Contracts 



Percentage of 


Number 


Percentage 




Contract Amount 


of 


of Total 


Total Amount 


Earned 


Contracts 


Contracts 


Disencumbered 


1 00% 


18 


8.8% 


$ o 


90-99 


24 


11.7 


277,000 


80 - 89 


33 


16.1 


864,000 


70 - 79 


30 


14.6 


2,569,000 


60 - 69 


13 


6.3 


1,532,000 


50 - 59 


16 


7.8 


2,526,000 


1 - 49 


51 


24.9 


18,712,000 


0 


20 


9.8 


4,345,000 


Totals 


205 


100% 


$30,825,000 


Some Panel Fiscal Practices 






May Not Be Appropriate 







During our review of the panel's fund allocation and 
disencumbrance processes, we identified a practice that may 
not be appropriate. This practice extends the life of the panel's 
fiscal appropriations. In fiscal year 1985-86, the Legislature 
changed the length of time for which funds were appropriated, 
or made available, to the panel from a "continuous 
appropriation" to an appropriation that is available for three 
years. Continuous appropriations allowed funds to remain 
available for use without regard to the fiscal year. Therefore, 
funds appropriated to the panel in a certain fiscal year not 
encumbered or spent by the end of that year would roll over 
to the next fiscal year and be available for other projects. 
Under the 1985-86 legislation, appropriations available for 
three years must be encumbered in the first year and spent 
within the next two years. If it does not spend the funds within 
these time frames, they revert to the originating fund. By 
changing the length of time it has to use its appropriation, the 
Legislature effectively forced the panel to be more accountable 
for its funds. 
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Although the Legislature 
appropriates training 
funds for only three years, 
the panel's practice of 
reallocating unspent 
funds extends this by at 
least two additional 
years. 




No panel funds have 
ever reverted to the 
Unemployment Insurance 
Fund. 




However, a provision in the annual budget act permits the 
panel to reallocate funds that it has disencumbered from 
prior-year training contracts. This provision allows the panel to 
change prior-year appropriated funds into current-year funds. 
For example, assume the panel encumbers $100,000 to a 
contract in fiscal year 1994-95 using funds appropriated in that 
year. In fiscal year 1996-97, it determines that the contractor 
will not be able to earn $25,000 of the contract amount. The 
panel can disencumber the $25,000 and re-encumber it for 
another project beginning in 1996-97. When the panel 
disencumbers and then re-encumbers funds, it is able to extend 
the length of time it uses the appropriation. In the above 
example, the panel and its contractors should have used the 
$100,000 originally encumbered in fiscal year 1994-95 
by the end of fiscal year 1996-97. However, the $25,000 it 
disencumbered and immediately re-encumbered in 1996-97 
will now be available for an additional three-year period ending 
fiscal year 1998-99. Figure 3 shows that, by disencumbering 
the $25,000 and then immediately re-encumbering those same 
funds, the panel is able to extend from three to five years the 
length of time the funds are available. 

We also found that the panel is using this budget act provision 
to prolong the availability of funds encumbered to particular 
contracts. Specifically, near the end of each fiscal year, the 
panel analyzes contracts appropriated with funds that will revert 
to the Unemployment Insurance Fund due to the three-year 
time limit. The panel disencumbers these funds and then 
immediately re-encumbers them to the same contract. It is thus 
able to change the appropriation identity of the funds to the 
current year, thereby extending the amount of time they are 
available to the panel. In fiscal year 1995-96, the panel used 
this provision to extend the availability of $1 5.4 million from the 
fiscal year 1993-94 appropriation. 

The statutes are not clear as to whether the Legislature, when 
it included this provision in the budget act, intended the 
panel to have the authority to perform these actions. However, 
by giving it this authority, the Legislature has allowed the 
panel to prolong the life of its funds indefinitely. In fact, no 
funds have ever reverted to the Unemployment Insurance Fund. 
The practice of disencumbering and re-encumbering funds is an 
inefficient process, whether it results from unsuccessful training 
projects or efforts to prolong the appropriated life of funding. 
To a certain extent, the panel cannot control the unsuccessful 
completion of a contract. However, to disencumber funds to 
extend the life of appropriations appears improper and should 
be discontinued. 



Figure 3 

The Panel’s Ability To Extend 
Length of Appropriation 



Fiscal Year 

1994-95 1995-96 1996-97 1997-98 


1998-99 


$100,000 encumbered 

Original threeryear period^i|-Nni|y^ 

$25,000 

Disencumbered 




1 

$25,000 


Re-encumbered 
New three-year per 


»o4 . 



The Panel May Be Spending More 
Administrative Funds Than Intended 




The panel is able to 
exceed its 15 percent 
administrative cost cap by 
charging unspent 
prior-year funds again in 
subsequent years. 




The practice it employs to prolong the life of its funds may 
also lead to the panel exceeding the amounts allowed 
for administration. Section 10206(a)(2) of the Unemployment 
Insurance Code states that the panel's administrative costs shall 
not exceed 15 percent of the amount annually available for 
expenditure excluding expenses related to marketing, research, 
and evaluations. The annual budget act determines how much 
will be available to the panel for expenditure. However, as 
noted in the previous section, when the panel disencumbers 
and re-encumbers funds, it is able to change their fiscal year 
identity. When this happens, funds disencumbered from prior 
years' appropriations are included as amounts available for 
expenditure in the current fiscal year. Therefore, the panel is 
able to spend on administration up to 15 percent of the funds 
appropriated by the Legislature for a fiscal year plus up to 
1 5 percent of those funds disencumbered from prior fiscal years. 
The panel is thus able to use more than 15 percent of the funds 
appropriated to it for administrative expenses. 



For example, in fiscal year 1994-95, the panel spent 
$9.1 million, only 10.5 percent of the $86.9 million funds 
available to it that year, on administration. However, 
disencumbrances comprised $45.2 million of the $86.9 million. 
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If we do not include disencumbrances, which were funds 
originally appropriated in a prior fiscal year, when calculating 
the amount the panel used for administrative expenses, the 
percentage would be 21.8 percent rather than 10.5 percent. 
Because the panel can control to a certain extent the amount of 
funds it disencumbers, it is able to influence, outside the 
purview of the Legislature, the amount it can spend on 
administration. 




Although it committed 
$7 million for a contract, 
the panel never intended 
for the contractor to 
provide training 
services. 




The Panel Encumbered Funds 

Inappropriate^ > 

During our review of its contracting process, we became aware 
of a contract the panel made with the California Manufacturers 
Association (CMA) that set aside funds for a specific group. In 
fiscal year 1993-94, the panel entered into a training contract 
for $7 million with the CMA. Accordingly, the panel 
encumbered $7 million of fiscal year 1993-94 funds for the 
contract. However, it never intended the CMA to provide 
training services. Rather, its responsibilities under the contract 
included marketing the panel program to the CMA membership, 
developing projects, providing technical assistance to interested 
companies, and administering projects approved by the 
eight-member panel. Effectively, the panel earmarked 
$7 million in training funds for CMA members by encumbering 
these funds in the CMA contract that did not involve training 
services. 

The State Administrative Manual, Section 1205, states that a 
contract will be comprised of the following elements: 



• A clear and accurate identification of the parties; 

• The time for performance or completion of the contract; 

• Amount to be paid; and 

• A clear and complete statement of the work, service, or 
product to be performed, rendered, or provided. 

The panel's contract with the CMA does not fulfill these 
requirements. As stated above, the panel never intended the 
CMA to provide training services. In addition, it has not paid 
any funds to the CMA; rather, when a company affiliated with 
the CMA received approval for a training project, the panel 
funded the project using current-year funds and then 



3 

ERIC 



A (L 



IF 



s • RililS 






29 



21 



disencumbered the same amount from the CMA contract. 
According to the executive director, the panel does not plan to 
enter into any additional contracts that are similar. 

We also noted that, at June 30, 1996, the panel had 
$3.9 million remaining encumbered but unused on the CMA 
contract even though the contract had expired on 
December 31, 1995. The executive director stated that the 
panel retained the $3.9 million encumbrance in April 1996. 
This was done to maintain funding for projects being considered 
by the CMA prior to the end of the fiscal year. However, the 
panel was not able to provide us a list of these projects. In 
August 1996, after we raised concerns about the contract, the 
panel disencumbered the remaining $3.9 million. 




Because of the 
inappropriate contract, 

$1 million in training 
contracts were delayed 
and $2.9 million was not 
reverted to the 
Unemployment Insurance 
Fund. 




The Unemployment Insurance Code, Section 161 1(b), states that 
any moneys allocated to the panel in a fiscal year and not 
encumbered in that fiscal year shall revert to the Unemployment 
Insurance Fund. Because the panel's contract with the CMA did 
not involve training services, the funds should not have been 
encumbered. At June 30, 1996, the panel had $1.0 million in 
contracts approved but not funded because it had run out of 
money. If the panel had not retained the $3.9 million on the 
CMA contract after the contract had expired, it could have 
avoided a delay in funding the $1.0 million in contracts on its 
waiting list. Moreover, because it did not have any other 
approved contract waiting, the remaining $2.9 million would 
have reverted to the Unemployment Insurance Fund. Finally, 
by processing a contract for services it never intended to pay for 
or receive, the panel wasted the time and resources of its staff 
members, and the staff at the Employment Development 
Department who handle the panel's accounting function. 



Conclusion 



The panel is accomplishing many of its responsibilities in both 
its program and administrative areas. However, it could 
improve its strategic planning efforts by stating its goals and 
objectives in measurable terms, and its units could better 
measure progress towards meeting those goals and objectives if 
they established performance benchmarks. In addition, the 
panel could be more effective and efficient in the way it delivers 
training funds to eligible California companies. Specifically, the 
panel needs to develop better estimates of amounts it awards to 
contractors. In addition, it needs to better monitor the amount 
of training funds its contractors earn. In making these 
improvements, the panel can maximize the use of funds 
available and lower the rate of disencumbrances it incurs each 
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year. Finally, the panel has engaged in certain administrative 
practices that, although legal, appear to be improper, including 
extending the life of its appropriated funds, expending more 
than 1 5 percent of appropriated funds for a particular year on 
administration, and encumbering funds to a contract from which 
it received no training services. 



Recommendations 



To ensure that it is successful in achieving its stated goals and 
objectives and fulfilling its mission, the panel should do the 
following: 

• State its organizational goals and objectives in measurable 
terms whenever possible. 

• Establish performance benchmarks for its employees so it 
can measure their success and challenge their performance. 

To reduce the amount of disencumbrances it incurs, the panel 
should take the following actions: 

• Delineate in its annual report to the Legislature the effect 
the panel's implementation of recommendations from the 
disencumbrance report has had on the disencumbrance rate 
when these effects are measurable. 

• If the changes it made as a result of the disencumbrance 
report are not effective in lowering the rate of 
disencumbrances, consider alternative means, such as using 
phased contracts, instituting a penalty on unsuccessful 
companies, or implementing other procedures suggested by 
the Advisory Research Council or the California State 
University, Northridge, study. 

To ensure that the panel is spending its funds appropriately, the 
Legislature should do the following: 

• Consider the methods by which the panel is allowed to 
extend the life of its appropriated funds, and if these 
methods are contrary to the intent of the Legislature, 
eliminate the budget act language that allows the panel to 
re-encumber funds. 

• Consider the methods employed by the panel to use more 
than 15 percent of appropriated funds for administration, 
and if those methods are contrary to the intent of the 
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Legislature, limit the panel to using only a certain 
percentage of each year's appropriation and excluding 
amounts disencumbered from prior fiscal year 
appropriations. 



To be more effective in its operations, the panel should stop 
entering into contracts and encumbering training funds for 
which it does not intend to receive training services. 



We conducted this review under the authority vested in the state auditor by Section 8543 et seq. 
of the California Government Code and according to generally accepted governmental auditing 
standards. We limited our review to those areas specified in the audit scope of this report. 

Respectfully submitted, 



State Auditor 

Date: October 1, 1996 

Staff: Elaine M. Howie, CPA, Audit Principal 

Dale A. Carlson, CGFM 
David E. Biggs, CPA 
George Alves 
Jennifer Buck 
Chris Harris 
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— — State of California — 

Employment Training Panel 



Pete Wilson September 23, 1996 
Governor 



Thomas C. EUick Kurt R. Sjoberg, State Auditor 
Chairman Bureau of State Audits 

James L. Quillin 660 J Street, Suite 300 
Vice Chair Sacramento, C A 95814 

Clifford Cummings 

Member j) ear Mr. Sjoberg: 

David A. Lattanzio 

Member y OU f or the opportunity to review and respond to the audit of the Employment 

Yvonne Nix-Everett Training Panel (ETP) program conducted by the Bureau of State Audits, as required 
Member t»y Unemployment Insurance Code Section 10206.5. The Panel commends your staff 
Patricia A. Noyes for the thoroughness of their, review and the manner in which they conducted this 
Member aud j t . \y e appreciate their courtesy in working with ETP staff and their professional 
Regina Render approach in reviewing the Panel program. Their willingness to learn the details of our 
Member program i s reflected in the clarity and quality of the final audit report. We read the 

Pa* Wil liams report with interest and are happy to provide this response to it. 

Member 

We are pleased the audit found the Panel is fulfilling it responsibilities in both program 
and administrative areas. The Panel will continue to ensure it fulfills these 
responsibilities in its role as a significant economic development tool for California. 
We will co ntinue to work closely with other entities such as the Trade and. Commerce 
Mores e. Mangine Agency, educational institutions, other State departments and the Panel’s Advisory 
Assistant Director Research c ounc ii and through ETP staff quality improvement teams, to ensure 

Peter G. DeMauro 

General Counsel continuous program improvement. 



Gerald G. Geismar 
Executive Director 

James M. Bratt 

Assistant Director 



ETP (2/96) 



Central Office : 

11 00 J Street. 4th floor 
Sacramento. CA 95814 
(9 16) 327-5640 
FAX (9 16) 327-5260 



Regional Offices. 

1 1 00 J Street. 4th floor 
Sacramento. CA 95814 
(9 16) 327-5640 
FAX (9 16) 327-5260 

4640 Lankenhim Blvd., 
Suite 311 

No. Hollywood. CA 91602 
(818) 505-6200 
FAX (818) 761-6409 

177 Bovet Road. Suite 180 
San Mateo. CA 94402 
(415) 573-3830 
FAX (415) 573-3857 

5333 Mission Center Road, 
Suite 300 
San Diego. CA 92108 
(619)686-1920 
FAX (619) 686-1921 



Generally, we concur with the findings submitted in the audit report. We agree that 
improvements can be made in several areas identified in the report. At the same time, 
many of these issues had also been identified by Panel staff and work is underway to 
make the identified improvements. The following provides our response to the specific 
findings of the audit report. 

Finding: The Panel Is Accomplishing Man y of It s_ ResP QPSibUitfeS, bwt C a n 
Improve Its Strategi c Planning Process 

Response : Again, we are pleased the audit found the Panel is achieving its overall 
mission through the appropriate application of its contracting procedures, program 
audits, fiscal processes, administrative procedures, and electronic data processing 
systems. We will continue to strive for excellence in these areas as we work with our 
many economic development partners to assure California businesses and workers are 
able to compete successfully in the global economy. 
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We recognize the need to set measurable objectives where possible in the Panel’s strategic planning 
process so the success of the program can be measured and continuous improvements can be made. At 
the same time, we feel the Plan, as written, meets the overall purpose of an ETP strategic plan as 
provided in the ETP legislation, and we believe it appropriately sets forth the Panel’s strategies and goals 
for accomplishing its mission. The Plan clearly delineates the Panel’s strategies for working in concert 
with entities such as the Economic Strategy Panel, educational institutions, other State agencies, and 
ETP’s Advisory Research Council to ensure skilled workers for California businesses in support of the 
State’s overall economy. 

In effect, the Plan is built upon several implicit measures which ensure the Panel is.able to continually 
evaluate the success of its program. ETP is unique in that its performance-based contracting requirement 
provides an inherent benchmark for measuring the success of the program at all times — i.e. r .a measure 
of success is provided automatically through an assessment of the number of trainees who complete Panel- 
funded training and the required retention period, vis-vis the cost of providing this training: The Plan 
is framed with the overall intent of continually improving the program’s ability to best meet these basic 
objectives. 

The Plan also sets forth the Panel’s research agenda for assuring ETP is meeting its objectives and 
fulfilling its mission, and to ensure the Panel addresses the State’s future economic needs. Here, a recent 
longitudinal study conducted by California State University, Northridge found ETP training has resulted 
in increased earnings and employment security for trainees, even three years after theicompletion of 
training and ETP training generated an annual return of up to more than five times the original 
investment. 

We feel the nature of performance-based contracting and ongoing program research and evaluation 
provide the Panel with effective means for determining the success of the program. At the same time, 
we agree that additional measurable objectives should be developed so the Panel can better state its 
organizational goals and establish benchmarks for its employees to achieve those goals. Therefore, where 
possible, we will include more measurable objectives in the Panel’s Three-Year Strategic Plan. 



However, given the primary audience for the Three-Year Plan are the Governor and Legislature, . we feel 
it is inappropriate for the Plan to include goals and objectives which are too detailed! Rather, such goals 
and objectives are more appropriately the function of an internal plan for implementing the broaderer 
strategies and objectives stated in the Three-Year Plan. Therefore, I have instructed, the Panel’s 
Executive Director to develop such an implementation plan with detailed goals and objectives by the end 
of the 1996 calendar year. This plan will be updated on an annual basis as an ongoing means of 
implementing the Three- Year Strategic Plan. 



Finding: The Panel Needs to Better M anage Its Distribution of Funds 

a. The Panel Needs to Better Estimate the Amount of Funds Contractors Will Earn 

Response: Since its inception ETP has been a 100 percent performance-based contracting program, 
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which ensures that the training is tied to a real job. This means no funds are earned by a contractor until 
a trainee completes all of the training and a subsequent employment retention period of at least 90 days 
in a training-related job at a stipulated wage. The previously referenced three-year longitudinal study also 
found, in terms of increased worker earnings, the Panel’s unique performance-based contracting produces 
outcomes similar to those found for training which is purely private. Researchers found this level of 
efficiency is unusual in a publicly funded training program. 

Further, ETP is an employer driven program. Businesses identify their own training needs and the Panel 
works directly with those businesses to help them meet those needs. Ideally, a successful contract would 
accomplish all training as planned, with a contractor earning all of the encumbered funds. However, with 
a 100 . percent performance-based contracting program, and with circumstances often beyond the 
contractor’s control, some fund disencumbrances will occur in most cases. 

With this imnind, the*. Panel has implemented a number of policies and procedures to increase the 
potential for successful contracts. During the develppment phase of the ETP contracting process. Panel 
development staff work with contractors to identify and reduce factors which could subject the training 
project to greater! risk of failure.. Panel monitoring staff also work to identify any problems at the early 
stages of projects so that actions can.be taken to correct the problems or terminate the contract and 
disencumber the funds. In addition, throughout, the term of the agreement the monitoring analysts 
continually assess the need for amendments to disencumber funds that will not be earned. 

b.'By Revising its Contract Management Practices, the Panel Could Serve Employers Sooner 

Response: The. Panel has taken action on the recommendations of its internal disencumbrance study, by 
imple menting policy and procedures for its staff. It will be some time before the effect of implementing 
the new policies and procedures can be measured- However, the. Panel will evaluate the effect of these 
actions on reducing disencumbrances and report the results of the evaluation to the California Legislature 
in its annual report. It will also report the results and findings from the California State University, 
Northridge study on the role of consultants in ETP contracts in the annual report. 

The Panel has taken the : following action on: the . remaining two recommendations from the 
Disencumbrance Study: 

• To encourage , contractors to plan their projects more realistically the Panel has implemented 

procedures to allow contractors to amend agreements to add funds at a later date without being 
required to. make a substantial. contribution: • . -• 

• To strengthen a contractor’s understanding of ETP’s contractual requirements and their commitment 
to -implement a successful training contract the Panel has also implemented a policy requiring an 
operational senior management representative participate in an initial site visit meeting conducted 
prior to contract development. 



c. The Panel Could Adopt Policies to Deter Contractors from Overstating Their Funding Needs 

Response: The Panel will consider the possibility of penalizing contractors who don’t successfully 
complete their training agreements, if necessary, after its careful and thorough evaluation of the effect 
the new policies and procedures have had on reducing disencumbrances. 

Finding; Some Panel Fiscal Practices Mav Not Be Appropriate 

Response : The audit report questioned the appropriateness of the Panel’s ability to change prior-year 
appropriated funds into current-year funds, thereby extending the length of time the funds are available 
(e.g., from three to five years.) We feel Budget language clearly allows the Panel to extend the life of 
budget appropriations and prolong the availability of funds encumbered to particular contracts. Language 
in the Budget Act, Chapter 162, 5100-001-0514, Provision 1, states “funds disencumbered from the 
Employment Training Funds training contracts ... are hereby appropriated for transfer to and in 
augmentation of, this item for allocation by the Employment Training Panel for training contracts”. 

However, we would point out the disencumbrance and re-encumbrance of funds of projects which have 
not completed occur only when it has been determined by Panel staff that the contractor in question will 
earn additional monies, but will not be able to invoice for those earnings until after the end of the fiscal 
year. Such delays in invoicing may be caused by : 

- the term of the contract not ending until the end of the fiscal year (i.e., late June), with closeout not 
due until 30 days after the end of the term (automatically carrying over into the next fiscal year); pr 

pending appeals submitted for trainees denied payment because of performance which was not met. 

Based on the fact the Panel’s training programs are performance based and most of ETP’s contracts are 
written for a two-year term, it is virtually impossible to control the level of success of contract 
completion. ETP enters into agreements with contractors with the assumption each contractor will 
provide the training the contractor has requested during the development of the contract, and all funds 
encumbered will be earned. Due to a variety of unforeseen circumstances related to the contractor’s 
business conditions, such as changes in the training needs, changes in production demand and equipment, 
employee layoffs, and inability of the employer to release employees for training, the amount actually 
earned by the contractor can differ greatly from the planned amount. Because of these factors, the Panel 
cannot control the amount of disencumbrances which can occur. However, the Panel has taken steps to 
help contractors improve their planning for achievable training, and to identify and reduce certain risk 
factors which can lead to higher disencumbrances. 

Finding; The Panel Mav Be Spending More Administrative Funds Than Intended 

Response: The audit report stated that the Panel may be spending more administrative funds than 

intended. It states because the UI code includes prior year disencumbrances in the amount annually 
available for expenditure, the Panel is able to influence the amount it can spend on administration. 
However, it is our view the Panel is spending the appropriate amount of funds on program administration 
and that amount is necessary to provide the needed administrative support for the associated training 
projects. 
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Unemployment Insurance Code Section 10206, (a) (2) states “The panel’s administrative costs..., shall 
not exceed 15 percent of the amount annually available for expenditure by the panel”. It is Panel’s 
interpretation the “amount annually available for expenditure” includes monies disencumbered from prior 
years. This seems to be confirmed by the fact such funds must be “available for expenditure” , since they 
are approved for and encumbered in training contracts during the year in question. Further, EDD which 
is responsible for administrating ETP’s budget, holds this same view. According to the Panel’s EDD 
budget analyst, it is also EDD’s interpretation that prior year disencumbrances are factored into the 
amount annually available. 

Further, we feel it is critical the annual 15 percent administrative funding amount be based upon this 
interpretation. These administrative funds are in direct support of the funds expended for training 
projects during the same fiscal year. All contracts approved during a fiscal year have associated 
a dminis trative costs which are distributed over the lifetime of a contract: These include costs for project 
development, contract processing, monitoring and technical assistance, fiscal support, project close-out 
and auditing. 

Finding: The Panel Encumbered Funds Inappropriately 

Response: The P ane l concurs with the auditor’s finding, but would like to explain the intent of the Panel 
at the time it entered into the master agreement with the California Manufacturers Association (CM A). 

CMA’s membership is made up of more than 800 major California manufacturers and food processors 
whose employees constitute over 70 percent of the industrial workforce in the state. The intent of the 
Panel was to serve the CMA membership by ensuring training funds would be available to retrain their 
current workers, or train new employees of companies expanding in or relocating to California. The 
resulting tr aining agreements with the CMA membership did meet ETP statutory, regulatory and policy 
requirements. 

The Panel agrees with the auditor’s recommendation, and will not encumber funds for, or enter into 
agreements with contractors who do not intend to provide training services. 

In conclusion, we appreciate the opportunity to review the audit report and provide this response. While 
we recognize improvements can be made as identified by the audit and covered in our response, we are 
pleased the audit found the Panel is fulfilling its program and administrative responsibilities. Again, we 
commend your staff for the professional manner in which they conducted the audit and for the quality of 
the final report. 

If you have any questions or would like to discuss these comments, please contact Gerald Geismar, ETP’s 
Executive Director, at (916) 327-5246. 



Sincerely, 




^ Thomas C. Ellick, Chairman 
cc: Gerald G. Geismar 
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cc: Members of the Legislature 

Office of the Lieutenant Governor 

Attorney General 

State Controller 

Legislative Analyst 

Assembly Office of Research 

Senate Office of Research 

Assembly Majority/Minority Consultants 

Senate Majority /Minority Consultants 

Capitol Press Corps 
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